Multimedia streaming—the continuous delivery of synchronized media data like video, audio, text, and animation—is a critical link in the digital multimedia revolution. Today, streaming media is primarily about video and audio, but a richer, broader digital media era is emerging with a profound and growing impact on the Internet and digital broadcasting.
Synchronized media refers to multiple media objects that share a common timeline. Video and audio are examples of synchronized media—each is a separate data stream with its own data structure, but the two data streams are played back in synchronization with each other. Virtually any media type can have a timeline. For example, an image object can change like an animated .gif file: text can change and move, and animation and digital effects happen over time. This concept of synchronizing multiple media types is gaining greater meaning and currency with the emergence of more sophisticated media composition frameworks implied by MPEG-4, Dynamic HTML, and other media playback environments.
The term “streaming” is used to indicate that the data representing the various media types is provided over a network to a client computer on a real-time, as-needed basis, rather than being pre-delivered in its entirety before playback. Thus, the client computer renders streaming data as it is received from a network server, rather than waiting for an entire “file” to be delivered.
The widespread availability of streaming multimedia enables a variety of informational content that was not previously available over the Internet or other computer networks. Live content is one significant example of such content. Using streaming multimedia, audio, video, or audio/visual coverage of noteworthy events can be broadcast over the Internet as the events unfold. Similarly, television and radio stations can transmit their live content over the Internet.
A U.S. patent application entitled “Multimedia Timeline Modification in Networked Client/Server Systems,” filed Sep. 15, 1998, Ser. No. 09/153,664, by inventors Anoop Gupta and Nosakhare D. Omoigui, describes a system that allows a user to vary the playback speed of streaming multimedia content using time-scale modification technology. A U.S. patent application entitled “Timeline Correlation Between Multiple Timeline-Altered Media Streams,” filed Sep. 15, 1998, Ser. No. 09/153,749, by inventors Anoop Gupta, Nosakhare D. Omoigui, and Liwei He, describes a system that utilizes time-scale modification so that a user can vary the speed of streaming content without destroying its intelligibility.
Although such systems allow a user to change the playback speed of streaming multimedia content without destroying its intelligibility, such changes are not always “seamless”. There may be rather lengthy delays, from the user's perspective, between the user's request for a new playback speed and the actual rendering of the data at that requested speed. Additionally, there may be breaks in the data where the user is presented with either a “paused” view of the streaming data or no data at all until the system is able to render the stream at the requested speed. These problems degrade the overall user experience in playing back the multimedia content.
The invention described below addresses these problems, reducing delays and breaks when switching between different playback speeds of time-scale modified streams.